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ABSTRACT 



This paper presents and discusses the results of a project 
in which seven teams of elementary school students were involved in designing 
and implementing interactive multimedia resources in science for younger 
children. During a period of 3 months, students in grades 5 and 6 were 
divided into teams and met regularly to program the multimedia applications. 
Findings indicated that students improved significantly in their science 
understanding and programming skills. The benefits and problems of 
integrating science content with multimedia design are discussed, as well as 
reasons why the quality of dynamic and interactive components in students' 
multimedia productions proved to be a better indicator of students' learning 
than the quantity of multimedia produced. Insights gained from these analyses 
will contribute to the design of multimedia tools and environments in which 
young designers create software for their own learning. (AEF) 
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We present and discuss the results of a project in which seven teams of elementary 
school students were involved in designing and implementing interactive 
multimedia resources in science for yoimger children. We foimd that students 
improved significantly in their science imderstanding and programming skills. We 
discuss the benefits and problems of integrating science content with multimedia 
design and discuss why the quality of dynamic and interactive components in 
students’ multimedia prodhctions proved to be a better indicator of students’ 
learning than the quantity of rnultimedia produced. 

Introduction 

In recent years there has been a surge of interest in the use of multimedia in 
educational computer applications. Developers have added sound, graphics, 
photographs, and video to applications that were previously dominated by text. 
Various research efforts investigate ways in which multimedia features, such as 
video (e.g., Cappo & Darling, 1996; Rubin, Bresnahan, & Ducas, 1996), graphics 
(Richer, 1994), or combinations of features (e.g., Jackson et al., in press), can 
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contribute to students’ learning. One underlying premise of most research is that 
providing students with multiple representational formats in a content domain helps 
to build their understanding and addresses the diversity found in students’ learning 
and thinking approaches. 

While these multimedia features potentially add value to applications, less attention 
has been given to approaches that provide learners with the opportunity to design 
and program their own multimedia software applications. To take a closer look at the 
potential of multimedia design for learning, we examined the cases of seven design 
teams that were involved in creating interactive multimedia resources about 
astronomy. During a period of three months, a class of elementary school students 
(Grades 5 and 6) was divided into teams that met regularly to program multimedia 
applications for younger students in their school. Previous research documented the 
learning benefits for programming and content matter for students (Harel, 1991; 
Kafai, 1995). In this article, we focus on the final multimedia artifacts created by the 
students. We examine various program functions and the multimedia content of the 
final software products. Furthermore, we investigate how the team products 
themselves reflect the contributions of their individual authors. The purpose of these 
analyses was to illuminate some of the knowledge that young students have about 
multimedia design — as expressed in their products and in their interface 
design — and in which ways designing multimedia afforded learning opportunities 
for them. Insights gained from these analyses will contribute to the design of 
multimedia tools and environments in which young designers create software for 
their own learning. 

Theoretical Background 

Learning through desi^ng multimedia applications is part of a larger effort to 
provide children with rich learning experiences (Brown, 1992). It proposes software 
design as a new model of integrating programming with other learning that goes 
beyond learning programming for its own sake (Harel & Papert, 1991). Sof^are 
design requires more than just the mere production of programming code: students 
have to consider interface design issues, deal with content aspects, and create, 
debug, and maintain their programs. Learning programming is consequently situated 
within a larger learning context and integrated with other subject matters. In 
designing their own applications, students reformulate their Imowledge by creating 
and implementing external representations in their software. Multimedia features of 
software can facilitate this process, as they allow students to use and combine 
various representational formats in one medium. 

While knowledge reformulation is an important feature of learning through software 
design, personal expression of one’s ideas is another. With the increasing 
proliferation of commercial education and entertainment software, students have 
become accustomed to a software production level that makes extensive use of 
multimedia features. While students might have a wealth of experience in using 
multimedia applications, they have little knowledge about making multimedia 
software. Creating multimedia applications is a complicated and collaborative 
enterprise. In research and commercial contexts, groups of professional designers, 
programmers, and content specialists work together for several months (Lammers, 
1986). Several studies have used this approach to study the thinking and learning of 
young software designers (Blumenfeld et al., 1991). One series of projects focused 
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on children designing instructional software for mathematics (e.g., Hard, 1991; 
Kafai & Hard, 1991) and instructional games in science and mathematics (e.g., 
Kafai, 1995; Kafai & Yamall, 1996). While these studies made special use of 
programming as a vehicle to foster children’s learning of content, other studies used 
platforms such as hypermedia and authoring environments (e.g., Lehrer, 1991; 
Carver, 1991; Spoehr, 1995). In all of these studies, the researchers reported 
significant learning results. In this particular study, we decided to focus on the 
nature of the applications created by the students. We used the final computational 
artifacts as a starting point for investigating what kind of learning opportunities in 
science and programming was afforded by the design of an interactive multimedia 
resource and how this was related to individual students’ contribution to the 
multimedia product. 

Research Context and Participants 

The project took place in an urban elementary school that fimctions as the laboratory 
school site for University of California — Los Angeles. The participating classroom 
was equipped with seven computers; each was set up as a workstation for the seven 
table clusters. An additional seven computers were in an adjacent room and were 
mostly used for related Internet searches. 

An integrated class of 26 fifth- and sixth-grade students participated in this project. 
There were 10 girls and 16 boys of mixed ethnic background (19 Caucasian; 2 
Hispanic; 3 African-American; 2 Asian) ranging between 10 and 12 years of age. 
Students were grouped in seven teams according to the following criteria: one 
“experienced” designer (who had participated in the previous design project), mixed 
gender, and different academic success levels. One week before the start of the 
project, students were given an introduction to the main features of the MicroWorlds 
Logo programming environment. The assignment was to build an interactive 
multimedia resource about astronomy for younger students. Students worked 3—4 
hours per week on the project for a period of three months, spending 46 hours in 
total of which 23 hours were dedicated to programming. Science instruction and 
programming time were combined. 

We collected data fi'om a variety of sources: fieldnotes and videotaping of students’ 
interactions, students’ notebook entries about work assignments and distribution, 
daily logfiles of the software, interviews with team members, and pre- and posttests 
assessing students’ science and programming knowledge and their views on 
collaboration and project management. For this paper, we concentrate on the final 
software products, which were analyzed regarding the nature of their fimctions and 
the contributions of individuaLteam members. The main data comes fi-om a 
classroom activity conducted toward the end of the project in which each team met 
and determined which individuals should take credit for which parts of the final 
product (e.g., text, graphics, and animations). The following analyses focus on the 
screens that constituted the interactive multimedia resource about astronomy. 

Results 

All seven teams finished a multimedia information resource at the end of the project. 
Our project observations documented that the completion process was not an easy 
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one. For one, students had considerable problems in the beginning sharing work and 
computer resources, as each team consisted of three to four students each, but only 
one computer was available to all the team members. Furthermore, many students 
were learning Logo programming as they were designing their multimedia resoiuce. 
While this provided an authentic context for students’ learning, it also limited 
students’ expression in the beginning to simple page designs. Later on, students 
started including animations of the life cycle of stars or the planet movements for 
lunar and solar eclipses. As for its science inquiries, each team posed several 
questions, such as “Is there life on Mars?” or “What do we know about Black 
Holes?” which they intended to follow up in their science research. As the project 
progressed, these questions changed in focus and range. Some students reformulated 
their questions, while others added new ones that usually were more specific, such 
as “What is the Big Bang?” 

Students’ Learning of Science and Multimedia Programming. In terms of the 
effectiveness of the overall intervention, we found that the design project was 
successful as a vehicle for both science learning and Logo programming 
development. To assess students’ improvement in their knowledge of science 
content, we administered pre- and posttests in astronomy. Our preliminary analysis 
showed significant differences for the pre- and posttests in students’ understanding 
of astronomy (Pretest: 31.5, Posttest: 37.2, df 25, t: 5.65, p < 0.05). Students’ 
understanding of Logo also improved (Pretest: 13.3, Posttest: 18.6, df 24, t: 4.38, p < 
0.05). In addition, in terms of programming skills, many students started out at 
ground zero, knowing no Logo at all. These results are strong indicators of 
individual students’ improvement in science and programming. We also see the 
importance of understanding what kind of products students designed, thus we 
analyzed first the team efforts and then how students’ individual contributions to this 
effort were structured. 

Evaluation of Educational Multimedia Software: Team Contributions. Each 
interactive multimedia resomce consisted of a set of interrelated screen pages that 
were linked together with the help of buttons or clickable objects that could be 
activated through a mouseclick. Some of the screens had combined media elements, 
such as text and graphics, while others worked only in one medium, text or graphics. 
The number of created screen pages differed considerably for each multimedia 
design team; yet this was not a good measme of production value because screens 
differed in their fimctionality. We defined three categories of screens: content 
screens, quiz/feedback screens, and information/navigational screens. Table 1 
provides an overview of the distribution of the different screen fimctions. 

'.V 

Table 1. Distribution of Screen Page Functions 




5 



file:///D|/NECC/PROCEEDS/KAFAI/PROCEED.HTM 





CON 


AN 


IN 


QU 


Team 1 


17 


2 


4 


6 


Team 2 


15 


3 


3 


11 


Team 3 


9 


2 


3 


- 


Team 4 


12 


1 


6 


13 


Team 5 


14 


2 


2 


12 


Team 6 


23 


1 


6 


18 


Team 7 


11 


0 


2 


31 



KEY : CON = content 

screens; AN = animations; 

IN = 

information/navigation; 

QU = quiz/feedback 

Content screens represent some piece of knowledge about the field of astronomy. 
They can take the form of text, pasted pictures, drawings, or any combination of 
those three design elements. The larger category of content screens was also broken 
down further to specify content animation screens. Content animation screens (CA) 
contain animations or simulations that exemplified djmamic aspects of the solar 
system, such as the lunar eclipse, the life cycle of a star, or the effects of gravity. 
Only in one instance was the player given the possibility to set the parameters for a 
game-like animation (team 3). 

Many groups in the project decided to include Quiz/feedback screens to complement 
their multimedia resource. Quizzes usually asked questions about the content 
displayed elsewhere in the product but occasionally introduced new material. Most 
quiz screens contained one or two multiple-choice questions with buttons linking the 
user to feedback on his or her response. Feedback screens consisted primarily of 
simple pages exclaiming “right!” or “wrong!” in a very large font. Only in a few 
instances were users provided with additional information, as in the question “Can 
Martians Dance: Yes or No?” The answer page in either case replied “There is no 
right or wrong answer to this question, because we don’t know if there are Martians 
on other planets” (team 6)? . 

Another type of screen common to most group products was 
Information/navigational screens that provided information about the designers 
themselves or displayed the title of the software and subtitles of topic areas, such as, 
“This is the Planets Section!” Other screens contained buttons or turtles that linked 
to different topic areas and provided information to the user on how to navigate the 
software, such as a table of contents. The graphical arrangements on these pages 
differed considerably. While in a few instances students took advantage of a 
graphical representation of the solar system as an entrance to different planets, many 
others just placed a variety of buttons on the page. 
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To summarize the design efforts by the teams, we see that students made extensive 
use of multimedia. A pervasive feature of all software applications is that students 
tried hard to emulate commercial software models. A comprehensive analysis of all 
seven final versions of the educational multimedia resource showed that all of them 
used “point-and-click” as their main mode of advancing through the program. They 
provided menu options, had navigational features, such as title screens, 
introductions, content overviews, and final screen and credit messages. In the 
instructional component, students (with the exception of one team) provided limited 
positive and negative feedback and sometimes explanations. But students’ software 
did not provide the player with a “Quit” option, a feature that either many designers 
did not have time to implement or did not consider important. Furthermore, “Help” 
options were absent as well. Being beginning interface designers, the students were 
able only in a limited way to foresee their users’ needs. 

Evaluation of Educational Multimedia Software: Individual Contributions. 

While all students contributed to the final versions of their software, the levels of 
individual effort among the students in any group needed further examination. We 
decided to use the final artifacts themselves not only to document effort and equity 
but also to examine the ways in which individual contributions to each group’s final 
product have different affordances for product appearance, individual credit-taking, 
and learning benefits. To create a score commensurate with students’ experience, we 
created a “design differentiated score” by looking at the types of screens created by 
individuals in the context of their total contribution. For that purpose, we used the 
three types of screens defined above, content screens, quiz/feedback screens, and 
information/navigational screens. Student differentiated scores for each type of 
screen were created using the same values as the raw scores. To create 
design-differentiated-screen-type scores for each student, we added up all their 
points for each category of screen. Thus, a student with a raw score of eight screens 
might have four content, three quiz/feedback, and one information/navigational 
screens. 

After examining the differentiated scores (see Table 2), it appeared that they 
explained much of the deviation firom expectations that showed up in the objective 
scores. Looking at screen types seemed to be a much more viable measure of 
participation than total screens. Students who created the most content screens and 
content animations were the ones who displayed the most leadership, spent the most 
time at the computer, and showed the most developing astronomy loiowledge. In the 
group products for which the student with the most total screens was not the same as 
the most dominant and loiowledgeable student, we saw that the one with the highest 
raw score created mostly quiz/feedback and information/navigational screens. This 
finding is interesting not only tiecause it lends support for measuring individual 
performance and contribution by some deeper means than merely counting the 
number of pages created, but it also provides a window into which types of screens 
are valuable for what purposes. In the following section, we discuss how the 
different screen types afforded different modes of thinking and learning benefits, and 
served different personal goals. 

Table 2. Distribution of “Design Differentiated Scores” 
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Discussion 

Learning through designing multimedia applications provided a fruitful avenue for 
students to build their astronomy and prograrnrning knowledge as they worked 
together in teams. 

Learning Affordances of Different Multimedia Resource Functions. Working on 
different multimedia functions served different goals for the designers, in both 
science content and multimedia learning. While informational/navigational screens 
presenting tables of contents, title pages, or personal information about the designers 




8 



file:///D|/NECC/PROCEEDS/KAFAI/PROCEED.HTM 



may not necessarily contain science content, they have definite value in terms of 
technological fluency. Making tables of contents, main title pages, and topic area 
title pages requires taking the perspective of the user and considering what he or she 
would find most helpful in navigating the software. Navigational screens are not by 
nature devoid of content, as illustrated by the comparison of the solar system model 
title page and buttons title page shown earlier, but many of the 
information/navigation pages designed by students in this study had little or no 
content. The process of making such pages is relatively easy, because no animations 
are required, no astronomy research is required, and little programming is necessary 
other than linking buttons. 

Quiz/feedback pages afford several benefits fi-om a software design perspective. For 
one thing, the t^e of drill-and-practice quiz designed by most groups is a “quick 
and dirty” way to add interactivity to the software. In fact, when asked by 
researchers why they were making a quiz, the students often responded that it was 
because the users would want more interactive features. Making a quiz takes 
relatively little time, when all that is required is a series of questions with buttons 
leading to pages that say “right” or “wrong.” Another benefit is thus that students 
who felt that their contribution was too small were able to increase the number of 
pages they created by adding a quiz on some topic at the last minute. From a science 
learning perspective, making quizzes does not allow the designer to experience the 
maximum benefits of the software creation process. Most quizzes in this project 
were a disjointed set of questions with little or no relation to one another, followed 
by feedback pages that gave no explanations of the correct answer. The process of 
designing a quiz, such as the ones we saw, may not help the designer develop 
systemic understanding in science, because a quiz only requires that he or she know 
the right answers to a series of unconnected science facts. 

Content/animation screens are the most difficult to create, but they also afford the 
most learning benefits, both in terms of technological fluency and systemic 
understanding. Even the most basic kind of content page, one with only text and/or 
pictures, requires deeper levels of thinking about astronomy than the disjointed kind 
of knowledge represented by a quiz. Because students had to think about their users, 
3rd and 4th graders, they had to write all the text sections in their own words and 
explain astronomy phenomena in language that younger students would understand. 
This consideration discouraged such practices as cutting and pasting of text from the 
Internet or copying word-for-word from books. Opportunities for even deeper 
thinking about users, programming, and the subject matter are afforded by more 
elaborate content pages, which use animation to depict some astronomy 
phenomenon. Creating animations requires the designers to learn Logo beyond 
relying on the basic point-arid-^click controls of the MicroWorlds system because 
they have to direct the turtle in Complex patterns and series. Most importantly, 
animations afford remarkable growth in systemic understanding. They require the 
designer to comprehend something well enough to create a model of the whole 
phenomena, as opposed to merely possessing a descriptive understanding of “what 
happens.” 

Integration of Multimedia Development Within Science Learning. From the 
preceding analysis and discussion, it is clear that instructional multimedia 
development is a challenging enterprise for young software designers (as it is also, 
incidentally, for most professionals). Students had to deal conciirrently with learning 
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about astronomy and learning about multimedia software design and all its 
components (i.e., code writing, debugging, and maintenance). One could argue that 
the concurrent learning requirements might have hindered more than helped 
students. On the other hand, while schools traditionally introduce these subjects 
sequentially, we see a special synergy in learning them together at the same time. 

For one, it is difficult to imagine a course about multimedia design without students 
being engaged in designing media about something. Design has to be situated within 
a particular context or domain, so that designers as well as users can judge the value 
of their implementations. Fiuthermore, situated multimedia design provides an 
opportunity for the leamers/designers to approach subject matter from a different 
vantage point than their own. Instructional design, as implemented in this project, 
affords particular learning possibilities not only for the designated user of the 
instructional material but also for the instructional designer. We found that 
interactive and dynamic aspects of screen designs provided a good indicator for 
learning that integrated the programming and science. The development of 
instructional animations facilitated students’ better understanding of systematic 
aspects of science while it led them at the same time into more sophisticated 
programming and vice versa. 

Students’ Conceptions of Users. Designing the multimedia application provided 
students’ with some valuable lessons in interface design. Students’ experience as 
multimedia consumers did not necessarily lead them to factor in the user’s 
perspective into their designs. We found that students had difficulties assessing user 
needs or problems. Some comments made by the users at the end of the evaluation 
session also indicated the difference of perspective: while the designers were 
focused on providing a fun experience and getting through their demos, the users 
were distracted by the large number of spelling mistakes they found on the content 
pages. Some users commented on the graphics (often repurposed from Internet 
pages), which they liked but found “blurry”; one user added that this gave graphics a 
more realistic touch, referring most likely to the idea that pictures taken from 
hundred thousands of miles away cannot be precise. A general complaint by the 
users was that the software content was new for them. They suggested that the 
designers should create different entrance levels, or, as one user said, “I would have 
buttons like beginner, intermediate, and expert.” Another suggestion was to stay 
within the more specific space theme and to use labels such as “expert astronaut,” 

“in training,” and “ground control.” 

Conclusions 

Designing multimedia soffw^e presented students with complex issues, some of 
which became evident in the previous discussions. Students needed to coordinate 
multiple demands in designing and implementing their work. Based on our 
observations and analyses, we found that elementary students’ experiences and 
associated conceptions do not adequately prepare them for managing long-term, less 
well-defined design projects, which require more flexibility and iterative planning 
strategies. Furthermore, we found that students need more support in their 
collaborative work if the design situation is to be an effective learning context for 
each individual team member. It is obvious that the transition from a multimedia 
consumer to a multimedia producer is not an easy one. Yet the potential benefits 
make it a worthwhile learning experience. 
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